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SYSTEM FOR AUTOMATED GENERATION method provides for selecting some of those media elements 

OF MEDIA in response to a request for media programming, and select- 
ing a temporal organization for the media elements. 

RELATED APPLICATIONS However, the temporal organization is not dictated by the 

TTiis application claims priority from U.S. Provisioaal ^ selected ^formation that regarding each of the media ck 

Patent Application No. 60/042,564, filed Apr. 1, 1997, which '^^^^^.^^ system selects and orders the media ekments 

is hereby iacorporated by reference in its entirety. aocordmg to the data m the request, and according u. 

information, such as permitted transitions, regarding the 

t'lEKU OF THE INVENTION media elements. The system prevents a user from selecting 

10 individual media elements. The media elements are then 

This invention relates to a method and computer- assembled into media programming, 

implemented system for creation of audiovisual program- another aspect of the invenUon, a method is pro^'ided 

for verifying vie-wdng and comprehension of a unique media 

BACKGROUND OF THE INVENTION program. The method includes providing, in the unique 

15 media program, a unique sequence of cues. The method 

There have been recent substantial advances in the capac- includes receiving from a viewer of the unique media 

ity to design customized audio visual programs for specific program information relative to said cues, such as responses 

purposes from a library of existing video clips and audio to questions included on a questionnaire, or in rCvSponse to a 

elements. Customization of audiovisual programing is use- telephone call made by the viewer. The received information 

ful in many applicalionh. For example, in advertising certain 20 is then compared to the sequence of cues to determine 

products, and in particular automobiles, one promotional whether or not the program was viewed, and the level of 

technique is to prepare promotional videotapes which are comprehension by the viewer. 

sent to potential customers on their request. The desirability another aspect of the invention, a method of creating 
of customizing such videotapes to demographic or other audiovisual programming from stored audiovisual media 
characteristics of individual consumers arc of course sub- 25 elements is provided. In a first step, from a database con- 
stantiaL Health care practitionere and managed care entities taining information concerning the audiovisual media 
have begun to provide instructional videotapes to patients elements, certain audiovisual media elements are selected. A 
with information regarding manning of various diseases temporal sequence for the selected elements is designated, 
and conditions. Customizing of such information to the Transitions between the media elements are automatically 
disease and condition of the individual, and demographic 30 selected 
characteristics of the individual, such as age, income, edu- 
cational level, psychographic characteristics such as per- BRIEF DESCRIPTION OF THE FIGURES 
ceived ^veUness and willingness to change behaviors, and piG. 1 is a schematic diagram of a system according to the 
other factors, would be valuable for increasing the efifec- invention. 

tiveness of such video tapes in communicating the informa- 35 ^ ^ ^ schematic diagram mdicating information 

tion to the reapient. contained in a database according to the invention. 

In accordance with present technology, it is possible to FIG. 3 is a schematic diagram indicating the organization 

create and store a Hbrary of brief video clips, and provide a ^ database accordmg to the mvention. 

database of mformation regardmg these chps. However, m t^^^ a - x. .-j- j-^* i 

J **u ♦u * * u 1 1- j'* 4fi FIG. 4 IS a schematic diagram indicating software ele- 

accordance with the present technology, a human editor ♦ ■ * ^- + +u • ^ 

^ u- K 1 c - J- -J 1 1- J ments m the system accordmg to the mvention. 

must make the ultimate selection of mdividual clips, and ^ . J , , . . , , . 

provide the final editing decisions, creation and selection of J^^^ ! ^ ^ flow chart showing steps m the assembly of an 

transitions so that there is a smooth visual and audio decision hst m the method accordmg to the invention, 

transition between adjoining clips in the program, and ^ is a flow chart showing steps in the assembly of 

checking of the content of the clips to determine that there programming in a method accordmg to the invention, 

is proper coverage of the appropriate subject matter in an FIG. 7 is a flowchart showing steps in a method according 

appropriate sequence. Automating of this editing process to the invention of verifying viewing of a program, 

would malce possible substantial flexibility and new possi- DETAILED DESCRIFnON OF A PREFERRED 

bmties for creation of audiovisual programmmg. EMBODIMENT 

Once videotapes have been provided to the user, it is „ j, ^ „^ ^ 
difEcult to verify whether or not the user has viewed the . Rfe™S FIG. 1, Uiere is (fcpicted a system 10 accord- 
program. Even if the program has been viewed, the level of '°. '"'en'io^ System 10 indudes a computer 20 
comprehension is difficuU to assess. including processor 30. mput devices 40, output and display 

. . . devices 50 and memory devices 60. Computer 20 may be a 
It isaccordmgly an advantage of this mvention that the 55 conventional personal computer with suitable processor 
disadvantages of Hie pnor art may be overcome. ^^^,0^^. capacity and speed. Computer 20 corn- 
Additional advantages of the invention, and objects of the municates in a suitable manner with Hbrary 70 of audiovi- 
invention, will become apparent from the detailed descrip- sual clips 80. Library 70 may be stored on any suitable 
tion of a preferred embodiment which follows. storage medium of adequate bandwidth and retrieval time 
CT i\j[\A A Toc TKn/cvFTTfAXT characterfstics. At present, certain fixed or hard disk systems 
^ SUMMARY OF THE INVENTION ^^^^-^^ ^^^^^^^ performance to other options. Digitized 

According to a first aspect of the invention, a system and audiovisual cUps 80 are stored in Hbrary 70. Each audiovi- 

method of creating media programming are provided. A sual cHp is suitably identified, such as by the name of one or 

database is provided which contains selected information more files which make up the clip, 

about each of a large number of media elements. The media 65 Database, 

elements may be, for example, audiovisual clips, 'llie ele- Computer 20 is also suitably associated with a database 

ments themselves are maintained in a suitable h*brary. The 100. Database 100 contains unique identifying information 
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for each clip and has associated therewith additional infor- 
mation often arranged in a hierarchical manner. Referring to 
FIG. 2, this information will be referred to herein as a header 
110. Individual items of information within the header will 
be referred to herein as tags. The tag3 arc divided into two 5 
general categories, namely control tags 115 and content tags 
120. Content tags 120 contain information identifying the 
content of the clip. The content is often embodied in three 
general categories, namely content, i.e., information com- 
municated by the chp, as indicated by block 125, viewer i( 
profiles for viewers most likely to be receptive to the clip, 
indicated by block 130, and intended use of the clip, 
indicated by block 135. The content tags contain information 
defined by a suitable system of classification of information. 
For example, in a database of clips for the assembly of video i; 
programs concerning medical information, the content infor- 
mation may, in addition to other systems, such as for 
categories of content mentioned above, make use of Inter- 
national Classification of Disease codes. In databases 
directed to assembly of programs for other topics, other 2} 
coding systems are used for the content information. Fields 
may be provided to indicate technical complexity, specific 
items of information conveyed, demographic characteristics 
of actors in clips, style of script, and other suitable infor- 
mation. One or more fields may be provided within the 2 
content tag to define content. The system by which the 
content information is defined is preferably organized to 
provide for relationships among types of information. vSuch 
relationships may be levels of generality, and other subject 
matter relationships. 3 

The organizational structure of the database may be 
hierarchical, with each layer of hierarchy defining a specific 
set of organizational principles. Referring to FIG. 3, at the 
highest level the global behavior of the universe of elements 
is defined. This is equivalent to automating a 'look-and-feel' 3 
function for the entire unit of programming. A given stylistic 
approach 300 can be defined through the use of default 
values which will be employed, for example, in the treat- 
ment of transitions between adjacent clips. There may be 
defined a set or range of transition values, such as a range 4 
from hard-ait to 24-frame dissolve. In tbis example, the 
master look-and-teel level of organization might define 
hard-cui as the aesthetically optimal, and therefore the 
default, choice. Numerous other aesthetic or programmatic 
choices that affect the delivery and placement of media 4 
elements in the concatenated stream can be defined. 

Also at the highest level of organization, typically xised in 
on-line applications only, there may be provided the viewer/ 
user interface options which define the ways in which any 
given class and security level of user will be allowed to 5 
actively as well as passively interact with media assets. We 
will call this the INTERFACE LAYER 305. At this level of 
organization, the behaviors of ancillary assets such as pro- 
motional segmenth, information identifying the system, 
advertisements and news-flashes are defined. These assets s 
embody aesthetic, program or instructional design, as well 
as market-driven, or viewer defined behaviors. 

Immediately below this layer is preferably the meta- 
contcnt layer. This is called the PROGRAM LAYER 310. 
Here are defined the type of assets and the core content t 
descriptions of those assets. By way of example, the types of 
assets may be defined as training, informational, and enter- 
tainment assets. Examples of core subject matter would be 
"medical", at the highest level, "health management"', at a 
lower level, and "diabetes mellitus", at a still lower level. ( 

Next in the hierarchy is the instructional design layer, or 
TEMPI ATE LAYER 315. This layer is characterized by a 
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family of defining values which describe the range of the 
target audience in specific demographic and psychographic 
terms. Additionally, the overall outline of the subject matter 
is contained in this layer and is associated with demc^raphy 
where appropriate. These outlining functions arc secondary, 
however, to the temporal organizational templates embodied 
in this layer. Here the instructional designer, or interactive 
author, defines the preferred temporal modes of presentation 
of the universe of assets. For example, the instructional 
designer might define that the block of programming content 
called EFFECTS ON THE HEART is presented across three 
fundamental age groups, two levels of detail (summary/ 
cursory and in-depth), both gender specific gR)ups and four 
distinct ethnicity components. Within this multi- 
dimensional array of program assets, the instructional 
designer might also define that the material be presented in 
the preferred sequence of— INTRODUCTION TO THE 
HEART, IMPACT OF DIABETES ON THE 
CARDIOVASCULAR. VASCULAR STRUCTURES, 
EFFECTS OF DIET, EFFECTS OF EXERCISE, Q&A 
SESSION, SUMMARY. 

Below the instructional design layer are the smaller 
organizational elements which allow for elasticity in the 
specifics of the implementation of the temporal design. This 
is called the MODULE LAYER 320 and in special instances 
the SEQUENCE LAYER. Fundamental to this layer are 
weighting factors which control likelihood of asset use, and 
allow for the deployment of elements which are free to float 
temporally in order to accomplish certain transitions and 
effective deployment of those elements which are slave lo 
the temporality functions. These elements as a group arc 
shorter sequentially-patterned program elements of content 
which organize imder the temporality principles of the 
higher layer. The free floating elements may have various 
linking geometries or parameters at the opening and closing 
thereof. Such elements can be used to bridge elements that 
cannot themselves be linked because a direct link is either 
disallowed or would mvolve use of disfavored transitions. 
The lowest level of organization is that of the individual 

I media elements or assets themselves. This is called the CLIP 
LAYER 325. These elements carry lags which define their 
specific content, such as: DIABETIC HEART, LEFT 
VENTRICLE, DAMAGE TO, HYPERGLYCEMIA, 
MALE, AGE 50, TALKING HEAD. The first three content 

i tags will be noted as being in hierarchical order from most 
general to most specific. The next two are examples of 
demographic tags, and the final tag is a simple example of 
a tag denoting style. These elements also carry production- 
specific control tags, which, as discussed in more detail 

I below, define such characteristics as allowable exit/entrance 
transitions for both audio and video. 

It is important to note that. In the operation of the system, 
there is an mhibition layer between the dip and the search- 
ing mechanism. The inhibition layer assures that the system 

; does not include in the programming every clip that is 
responsive to a query provided by a user. The inhibition 
mechanism may be responsive to psychographic character- 
istics of the user, such as age, level of education, or even 
reason for the query. The tags arc responsive to this type of 

) information. The inhibition medianism may be modified 
dynamically as the database is mined for suitable assets. The 
inhibition mechanism may be viewed as in a multi- 
dimensional set of psychographic requirements. Clips can be 
evaluated for their responsiveness in the various dimensions. 

5 The system may set, or the user may select, a length of time 
for the program, and the inhibition mechanism will operate 
to limit the total number of clips selected to the selected 
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time, as well as choosing clips according to suitability for 3. A patient profile entry interface 415 and decision list 

tlie viewer and the viewer's purpose. processor. 

Referring again to FIG. 2, control tags define audio and 4. A decision/transition list editor 430 with built-in tran- 

video components of clips. Th& audio and video information sition previewer for both audio and video components 

may be entirely separate as indicated by block 140, VIDEO 5 of clips. 

CONTROL TAGS, and block 150, AUDIO CONTROL 5. a production player/recorder which embodies an 

TAGS. For example, a minimum number of control tags engine for both audio and video transitions, 

would be, for each of audio and video, transition points 145, Each of the foregoing components operates as an inde- 

155 an optimum transition point from the beginning of the pendent entity in the exemplary system. Each component is 

clip, a first possible transition point from the beginning of 10 started by the user selecting an icon from an appropriate 

the clip, a last possible transition point from the beginning folder. The user interface is preferably be graphical in 

of the clip, an optimum transition point from the end of the nature. Actual keyed data entry is preferably kept to a 

clip, a last possible transition pouil from the end of the clip, Tninimum where most entries are made by selecting an 

and a first possible transition point from the end of the clip. option from a set or an entry from a list. 

Each of these points may be defined in time, by video 15 xhe Production Interface 

frames, by tags of cUps that may be the subject of the xhe production interface 405 is the mterface designed to 

transition, or by content parameters. These points may be provide a structured, interactive means of importing a video 

defined by human post-production editors entering such cUp into the system. Astandard container, such as OMF, may 

information into the database in response to suitable prompts be utilized as a standardized vehicle for the transmission of 

contained in the softw^are. These points may also be defined 20 the proprietary tags. When selected, the user will be pre- 

by truth-table or expert-system defaults, which may or may sented with a screen containing fields for the entry of the 

fp--^ not reference either video or audio content. The variable following: 

Q beginning and exit points may be referred to as creating clip ID— This will become the primary key or file name. 

S extremities on the clips. This is believed to be a novel Source — ^Where the clip will be read from. Options may 

M feature as compared to existing databases of audiovisual 25 include DAJ' tape or network. 

IJI assets, and lends significant flexibihty to the process of The user will also be presented with one or more inter- 

m creation of transitions. Numerous optional control lags can faces to enter the audio/video transition fields and the coding 

be used, such as, with reference to video information, a (medical/patient demographic) information. Before being 

1^1- luminance range 160 for the opening and closing of the clip, allowed to exit the process, defaults will be applied to the 

Q preferred or requued transitions, including length and type 30 transition fields not specified by the user. The user may be 

IjI of transition, a dominant chroma value for each of the required to preview the clip just imported. 

opening and closing of the clip, start and end points for xhe Coding Interface 

'l^ dialog and action. With refer«ice to audio information, The coding interface 440 is the GUI designed to perform 

Q control tags may include key of music 170, tempo of music the entry of the medical and socio-demographic selection 

fi 175,prefen:edtransitionsandotherinfoimation.Thistypeof 35 codes which apply to the clip. These include the ICDs for 

information can be derived by suitable software reviewing which this clip is applicable, the socio-demographic and 

? 'f the data stream before or after the clips are concatenated, or medical attributes of those patients assumed to be the most 

hM advance of the assembly process using clip analogs or likely potential viewers of the clip, and any special fields 

Q proxies containing suitable information from the tags. which when present in the clienl/paLient profile will cause 

35 Creation of Database. 40 this chp to be selected. All fields may be selected from 

I '4 llie database is created by identifying each clip or other preestabHshed lists or groups displayed as '*'radio-buttons'\ 

asset and defining values of the control tags and content tags ^s the ICD set may require the designation of hundreds or 

for each. Values of the various control tags and content tags thousands of codes, a suitable method of use of the hierar- 

may be defined by a user, either during development of the ^hical nature of the ICD structure may be utiHzcd to simplify 

script for the clip or upon completion of the clip. For 45 the selection process. These may be selected during the 

example, the program may include screens prompting a user g^ript outlining or scripting process by the use of menus or 

to select a value for each control tag from a menu of options. existing fields incorporated in the scripting soft\^^are. 

Different screens may be provided for different users, such xhe Patient Profile Entrv Interface 

as dialog editors, audio editors, and video editors, to enter jhe Patient profile entry interface 415 is the GUI 

values for appropriate control and content tags. 50 designed to perform the entry of the general, medical and 

Alternatively, values of various tags may be created auto- socio-demographic codes for the specific patient for which 

matically by truth tables or decision -capture systems, or the product is to be produced. The general section may 

other automated techniques, either with or without human require the most data entry including patient name, address, 

assistance. Such systems may act from information derived }4t)dal security number, insurance billing numbers, referring 

from analysis of existing clips using hnage-recognition 55 entity identification and any text which is to be added to the 

software, from analysis of scripts entered into text, or from ficial video. Systems, such as those used in the health care 

other information provided by htmian editors industry, may be employed to extract relevant information 

By way of exa.mp]e only, a system for creation of a from existing patient records. In the case of on-line use, the 

database for use in connection with creation of medical medical and socio-demographic sections will be "click to 

progranuning will now be described witii reference to FIG. go select"' entry similar in nature to the entry process performed 

4. The system may include five primary components, each of by the coder when the clip is imported. The result of this 

which will be provided witii its own user interface: process will be to create an entry in the patient database 420. 

1. A production interface 405 which provides an interac- The information is then forwarded to the editor program 
tive means of importing a video clip mto the system and 425. 

recording the transition production information. 65 Based on the Lopic requested, such as a condition which 

2. A coding interface 410 to allow the entry of the medical the patient has been diagnosed with, and the demographic 
and patient selection codes which apply to the clip. information, editor program 425 will, as discussed below. 
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produce a recommended decision list, or preliminary edit Editor Program. 

decision list (ED L) file for further processing. Subsequently, Programs are created from clips by an editor program, 

the final EDL is created in an evolutionary manner. At first. The identify and order of the clips that comprise a 

a sample order of clips, without transitions, is analyzed. The program may be defined by a variety of methods varying in 

order and identities of the clips is then revised. When the 5 the degree of control exercised by the individual user. At the 

order and identities of the clips has been finalized, the level of least control by the user, the user defines only the 

transiti(^ns are computed. The transitions are then executed overall functional requirements for the program. For 

and mserted into the stream by use of a system of removal example, in the assembly of programs for education of 

of portions of the tag extremities used in temporal tiansi- patients about health issues, the user may define only the 

lions. demographic characteristics of the intended recipient of the 

The List Editor/Prcvicwcr program and the information to be conveyed, e.g., manage- 

The list Editor/Previewer 430 is the GUT designed to ment of a particular condition. Tn one embodiment, the 

provide the production editor with the ability to charge, add system may simply select appropriate clips from the data- 

to and preview die EDL that has been produced by the editor base and apply suitable transitions. Alternatively, an expert 

program 425 in response to information entered in the system included within the editor program selects and orders 

Patient Profile Entry Interface 415. one or more suitable templates as shown in block 505 of 

The Production Player FIG. 5. The template may incorporate the demographic 

Each chp, chp component, or other media asset, is stored characteristics and information to be conveyed, or may 

in three distinct segments: contain only sequential requirements for the underlying 

1. lead-in segment structures used in the program. Using the requirements in the 

2. main body 20 selected templates and the demographic characteristics and 

3. exit segment ^^^^^ requirements specific to the program, as well as the 
Transitions can only be performed on the lead-in and out inherent structural characteristics of the modules and 

segments. The production player 435 is launched with the sequences, the expert system or truth table then selects and 

name of the patient EDL to be played. The EDL will be orders one or more suitable modules and floater sequences, 

analyzed and each transition rendition time estimated. The 25 as shown by block 510 of FIG. 5. Using the requirements in 

production player will then process each cummand in the the selected modules and the inherent structural character- 

EDL. Most commands, which require very little processing, istics of sequences, the expert system or truth table selects 

cause a more detailed command to be written to the player and orders suitable sequences, as shown by block 515 of 

FIFO which controls the player output thread. Other process FIG. 5. Finally, using the requirements in the selected 

intensive commands, such as a transition, will be rendered, 3iJ sequences and the inherent structural characteristics of the 

stored to disc and the detailed command with its disc cHps, the expert system or truth table selects and orders 

location is added to the player FIFO, As each EDL command suitable clips, as shown by block 520 of FIG. 5 . The expert 

is processed, the remaimog estimated time flow is modified system or truth table also makes transition selections at the 

until the program determines that future transition rendering beginning and end of the audio and video portions of each 

will be able to be completed before the player output thread 35 clip, as shown by block 525 of FIG. 5. At any level in this 

requires it. At this point, the player output thread will be process, if there is an incompatibility, the expert system or 

started taking its input instructions from the player FIFO. Uruth table may discard an element and replace it with a 

The player output thread wiE operate asynchronously, trans- substitute element, or employ suitable transition methods, 

ferringeach data component .specified in the player FIFO Lo For example, if it is not possible to assemble acceptable 

the appropriate hardware component, such as video and 40 sequences in the manner dictated by the modules under the 

audio decoder modules, text merge processor, etc., whose template, the expert system or truth table may discard one or 

output drives the device for recording the program, such as more modules and select alternatives that meet the require- 

a videotape deck ments of the template. 

Upon completion, the required managerial and billing The tags may include information regarding such 

information will be generated and stored in an appropriate 45 attributes as luminance, chrominance, music tempo and key, 

database. and colors, at the beginning and end of each clip. 

The transition process will require the identification of Alternatively, a MIDI record of the audio portion of the clip 

both an exit and lead-in point, and a transition function may be analyzed. The editor program may apply expert 

specification. This pair of fields are refined into a single system software or truth tables to determine whether a direct 

recommended field as defined above, and then the final 50 transition between any two clips meets aesthetic require- 

recommended EDL is obtained. These fields are contained in ments. If not, the system may identify a suitable type of 

the preliminary EDL which is being played. The appropriate transition for both clips, add a bridge between the clips, or 

time positions are located in both the exit and lead-in determine that one of the clips must be discarded. For 

segments. The first frame of each segment is decoded into example, if the keys of the music in two adjacent clips are 

separate buffers and then the transition function is applied to 55 incompatible, the system may add a bridge consisting of a 

each matching pixel of both knages to produce the third burst of percussion in the audio portion of the program, 

rendered frame buffer. A similar process is also applied to The viewer or user may directly select templates, 

the matching audio sample points associated with the cur- modules, or sequences, depending on the degree of control 

rcntly rendered frame. The video and audio is then desired. Decisions rcgardb^ cither specific production dcd- 

re-compressed and written to the output transition file. 60 sions or global stylistic choices or content or order can be 

Values of certain control and content tags, such as key or captured by die system by, in addition to expert systems, 

tempo of music, may be determined by suitable software truth-table or other decision tree programming specific deci- 

from the clip. sion collection fields provided to content developers. Expert 

It is to be emphasized that the foregoing system is merely systems, truth tables and other systems may be used to create 

exemplary, and does not limit the scope of systems that may 65 the tags associated with the cHps. 

be implemented within the scope of the inventions described Use of expert systems, or decision-capture systems is of 

in this application. interest because the system might organically evolve stylis- 
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tic tendencies which might mimic or mirror those of creative 
professionals. Templates might be provided to content pro- 
viders or even to end-users, which would allow a specific 
style of editing, audio cutting or mixing, or program forma- 
tioi^ perhaps a template may be provided associated with an 5 
individual editor. Such decision-capture systems already 
exist for other uses which could be adapted to the assembly 
of audio and video. 

In one embodiment, the user creates queries depending on 
the qualities desired for the organization levels to be iden- lO 
tified. The editor program then identifies suitable templates, 
modules or sequences based on the queries. The relative 
importance of dilTerent items in the queries may be 
weighted. The weighting may be supplied by the user in 
response to suitable prompts, or may be generated automati- 15 
cally by the system based on existing programming. Using 
suitable relationships among data items, the user may be 
presented by the editor program with one or more templates, 
modules or sequences that represent a fundamental 
response, another set that represent a secondary response, a 20 
third set that represent a tertiary response, and so forth. 
Assembly Program. 

Once the set of cHps has been defined, the assembly of the 
clips from the library takes place. This Is accomplished by 
an assembly program. Referring to FIG. 6, the assembly 25 
program reads a file of clip identifiers and transition inlor- 
mation created by the editor program as shown by block 
605, and calls to the library for each identified clip as shown 
by block 610. The identified clips are then transmitted from 
the library to a suitable destination. The assembly program 30 
uses the transition information in assembling tbc clips into 
the program. If the clips are transmitted to a suitable digital 
to analog converter, the clips may be assembled into a 
program on videotape without compression. However, in 
order to assemble the program as a digital audiovisual 35 
program in memory, or to transmit the program over tele- 
phone lines or the Internet, data compression is required. 
Suitable data compression techniques and standards may, be 
used. 

ITie assembly program may also dictate particular tran- 40 
sitions For example, asymmetrical transitions in audio are 
advantageous. A leading edge audio transition which is 
exponential and short, and a trailing edge transition in audio 
which is linear and long is preferred. The video transitions 
need not match the audio transitions. 45 

The following is a technique which is believed by the 
inventor to be suitable for improving performance of com- 
pression of the programming, for use in compression 
formats, of which MPEG is an example, which achieve 
performance using predictive compression which includes 50 
both forward and backward lookuig firames. The technique 
is the elimination of the complex frames from the MPEG 
stream using the last-in and first-out video marker tags to 
determine the amount of leading and trailing videt) which 
might, in a worst-case scenario, participate in a visual effect. 55 
By eliminating these (P and B) frames, it is possible to 
employ a partial decoding of the (Huffiuan) algorithm, rather 
than a fiill decode/encode cycle. Additionally, the elimina- 
tion of these PB frames allows impunity in the concatcoation 
points employed. This freedom is bought at the price of 60 
increased bandwidth requirements, but only for anall seg- 
ments of video rather than entire clips. 

Another technique is applicable to compression formats, 
such as MPEG, where the chrominance data is subsidiary to 
the luminance data. This technique involves emulating the 65 
MPEG transitions by extracting luminance-only data. 
Because the chrominance data 'rides on* the luminance, this 
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can still be employed to generate useable dissolves. Using 
this technique, the fiill decode process can be further 
reduced, and thus accelerated, by processing this luminance 
in averaged blocks of pixels. These pixel blocks are 
employed in the encode process. Even without the use of 
MPEG, it is possible that some or all of these shortcuts might 
be effectively employed to accelerate the creation of dis- 
solves without use of a full decode/recode cycle. 
Viewer Database. 

Separately from the database of media assets described 
above, there may be provided a database of viewer infor- 
mation. This database defines the identity of each viewer, 
including name and address informalion, and may also 
include social, economic and medical data regarding the 
viewer, the history of items viewed by the viewer, and a file 
of preference information for the viewer. In an on-Hne 
environment, the viewer may be prompted to provide viewer 
identification information which may be entered into a 
database or may be retained temporarily as a viewer data 
record for use in creation of customized programming. 
Verification of Viewing. 

The verification of viewing of the delivered video pro- 
gram is difficult to accomplish. It is important for the 
well-being of the viewer when the program contains thera- 
peutic or training video programming, and useful for all 
content to assess the clTccliveness of delivering informalion 
via customized video progranmiing. An effective system 
would: (a) allow the distributor of the programming to know 
if no effort had been made on the part of recipient to watch 
the delivered material; (b) provide disincentives for fast- 
forwarding and skipping through the program; (c) allow for 
the confidential verification of the particular program 
watched, thus allow for the confidential generation of 
viewer-comprehension testing; (d) provide incentives, per- 
haps monetary or material, for the successful viewing of the 
material; (e) maintain confidentiality of the viewer and/or of 
the precise content of the delivered program. The following 
viewing verification system performs all of the objectives 
ouLlined above. 

Referring to FIG. 7, the program software creates an 
identifier sequence of cues, as indicated by block 705. The 
cues may be numbers, letters or other indicia, such as colors, 
shapes, sounds or combinations thereof, unique to each 
viewer and each program. The sequence may be derived 
from such information as the viewer's ID number, such as 
the Social Security number or other unique record locator 
contained in the patient/viewer record database, the unique 
segment and/or program ID numbers or the content of clips 
or other portions of programming delivered to the viewer. 
Regardless of the method employed, a unique key sequence, 
or unique sequence of cues, which may be alphanumeric or 
comprise a series of images, icons, colors or sonic cues, \s 
created, lliis sequence is unique to the citstomized video 
program. The elements of the sequence are recorded within 
the program, as indicated by block 710. The cue are pref- 
erably spread throughout the temporal sequence of the 
viewing experience one or more cues at a time. By way of 
example, numbers in the sequence may be superimposed 
over picture elements using character gpncration and keying. 
The picture sequences which underlie these characters may, 
or may not address the superimposed characters/numerals. 
In the case of audio cues, it is of course more likely that a 
specific sequence, such as a talking head, would be 
employed to communicate the cue. A special case of the cue 
is the use of actual questions, usually relating to viewer 
comprehension of the presented material and having mul- 
tiple choice responses. 
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As indicated by block 715, the viewer is provided with a is, at minimum, the verification of active viewing of the 
method for response. The response method inchides suitable temporal stream of information. The responses are corn- 
method to record these numbers, characters, colors, shapes pared to cues and expected responses, as indicated by block 
or other cues contained within the sequence. For viewers 725. At maximum, the information is also capable of veri- 
who arc receiving the programming online through a modem 5 fying time of viewing, comprehension of content, attitudes 
or Internet Unk, this might be a window which remains open regarding content (by asking for reactions in some or all of 
and active during the playing of the video sequence, or the fields), as well as motivating viewers i^ith the possibility 
which becomes open and active upon the presentation of the of prizes, secrets, entry into further lotteries or drawings, or 
appropriate superimposed strings, which allows the viewer other perceived rewards. The information can be tised to 
the opportunity to record that sequence. For videotapes 10 instruct viewers to view the material again, make calls or 
delivered to the viewer, a card may be delivered togeliier other responses to the system, view or read additional, 
with the videotape. The card may contain boxes to be supporting or further material. All of this, significantly, can 
checked, blanks to be filled in, or choices of opaque cover- be done while Tnaintaining complete viewer privacy. It may 
ings to be removed. Suitable instructions will be provided in also be important that some of the feedback can be local to 
the video for the viewer to scratch off or mark the card in an 15 the viewer, while other portions of the response-strings may 
appropriate place. involve system -collection of responses. 

In the case of scratch-off cards, or other pre-prepared Techniques may be used for distinguishing between incor- 

cards, it is significant that colors and/or shapes might be rcct responses resulting from a failure to view or comprc- 

employed rather than known characters, or as elements in hend the programming and an error in completing a response 

known characters, like the commonly employed segments of 20 card. Techniques may also be used for identifying the most 

characters used in LED displays. Some action is required of likely error in optical character recognition review of 

y^. the viewer to cause these characters/signs to be recorded/ responses. The sequence of numbers, letters, or other visual 

transferred to the recording area. These cards might bear or audio cues which is generated for and inserted in the 

y sponsorship or advertising data and this data might also programming is also used to generate computer-readable 

generate or contain further fields, lliese cards, or on-Une 25 printed information, such as bar code, corresponding 

\m components mi^t come irom a third-parly source. For uniquely to that sequence. Customized codes may be devel- 

instance the on-line window might be generated for, or by, oped to correspond to cues other than alphanumeric char- 

'''^1 an advertiser or vendor. A paper-based recording space acters. The printed information is applied to a remm card 

Q which might be contained in a booklet, magazine, newspa- supplied with a videocassette on which the programming has 

1^ per or as part of the packaging itself, may also be generated 3U been recorded. The printed information may be encoded, by 

!^ by or for an advertiser or vendor. Such a rccordmg space altering the order of codes or including additional codes or 

U% may be a peel-oflf videocassette label or a peel-off, tear-off, other known techniques, for security purposes. Correct 

u scratch-off, or stick-on component of the videotape packag- answers to question fields may be embedded in the bar coded 

jsss ing or its electronic analog. This component might also string. 

Jjf contain one or more other characters, colors, icons or strings 35 Acceptable tolerances for error within which the rcspon- 

19 which might be included in the final string, or used sepa- dent will be deemed to have viewed and/or adequately 

f ^ lately. This component might also be custom-generated as comprehended the programming may be determined for 

jaa| part of the custom packaging. For instance, the sequence each position of the character string. Numerous consider- 

35 ABC-might already occupy a place in the sequence- ations may be used in determining these tolerances. Exem- 

lj;J recording fields such as ABC-xxx-xxx-xxx. 'llie ABC-might 40 plary considerations are the following: (1) the first position 

be visible or might require manual intervention or problem of the string is prone to a heightened incidence of general 

* soMng to reveal. This problem solving might be related to user error; (2) fields which call tor the answering of ques- 

the material presented on the \ddeo. For example, a scratch- tions or scratching off of a layer of obscuring material are 

off grid of multiple choice questions, or an on-Hnc emulation subject to a heightened incidence of general user error; (3) 

of such a grid, presented by the programming or packaging 45 for positions known to include only a limited number of 

itself might yield a unique sequence of characters or signs. possible responses, statistical weighting can be employed to 

The final uses of this character string are manifold, but the assign unclear responses into one of the known responses, 

overarching intention is to p^o^'ide a motivational arena for e.g., if the possible responses are to w^rite out one of the 

the recording of this verification data. The verification data, capital letters A, B, C and D, and an OCR program reads a 

unless used in an on-line environment where the respondent 50 response as the numeral 0, the response will be assigned as 

is known to the system, usually contains fixed identifying the letter D; (4) in a system employing scratching off of 

data used to coniBrm the identity' of either the viewer/ selected areas on the face ofa response card, columns which 

respondent or the specific program viewed. It may also contain more than one removed field where one field is 

include, or consist solely of, data strings generated in correct and not mtjre than two fields are removed. A 

response to queries. It may further include gaming fields and 55 weighted tolerance system may also empby other strategies, 

or third-party sponsored information. For example, a parity field may be derived from a modulo- 

In the case of traditional videotaped or other passively cycUng of the total character string, where the base number 

consumed media, the response data can be collected swiftly of the modulus might be the number of total possible 

by 800-number, or standard toll-line, call-in to an automated characters potentially employed in a single field of the 

collection system, which has been programmed in a suitable 60 string. By way of further example, a system may be used 

manner to collect response data. The response data can also which causes an known and repeatable limitation to be 

be collected by mail-in and even correlation with sealed- placed on the possible use of a given character derived from 

envelope or other hidden message systems such as peel-open the earlier or other character in the string. The source of such 

and scratch off answer fields which provide interim instruc- a check character is from within the printed information, 

tions. For viewers with suitable computer equipment, the 65 whereas the checked characters are from the handwritten or 

response data can be returned by electronic mail. ITie receipt scratch-off fields. For example, the presence of an even 

of responses is indicated by box 720. The end result though number in the first position of a string might dictate the use 
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of an odd number in the second field. Alternatively, a 
template could be created which would dynamically or in a 
predetermined manner limit the possible characters or other 
cues which could be employed in each position of the 
generated string. This data can be used to optimize the OCR 
function of each position of the response string, as well as 
to dynamically shift the error tolerance on a position-by- 
positioQ basis. 

By creating a master parity diaracter and a known pattern 
in the string of characters, it is many times more probable 
that a single incorrectly read character can be isolated b> the 
logical cross-reference of the two strategies set forth above. 
For example, if the parity check is incorrect by one value, 
and the fourth position of the string should be an odd 
number, but is read as the numeral 8, it is likely that the 
fourth field contains the numeral 8 if it is also true that the 
fourth field cannot by design contain an even number. When 
such techniques arc combined with dynamically weighted 
general error correction, the scheme becomes tolerant of 
user inaccuracies and OCR errors. 

For the presentation of video -superimposed characters, a 
static template could also be used which matches an accom- 
panying printed template. Alternatively, video -generated 
characters could be derived dynamically from the character 
sequence used to identify ttie viewer. In this scenario, 
multiple-choice questions asked of the viewer would be 
presented on screen as would be the appropriate multiple- 
choice answers, but the characters used to identify the 
answers would be dynamically generated in response to the 
viewer or programming identification sequence. For 
example, if an algorithm generated the output X, then the 
characters A, B, C and D would be generated next to four 
multiple-choice answers appearing on screen. If the algo- 
rithm generated an output of Y, the answers would be 
assigned the numerals 1, 2, 3 and 4. Such a technique has the 
advantage of allowing the video assets containing test mate- 
rials to remain mutable while responding dynamically to the 
needs of the error-correcting,/checking system. It vslll be 
understood that the foregoing techniques may be employed 
with any visual, sonic or other cue. 

Various points of the systems that are believed by the 
inventor v*dll now be emphasized. However, the enumera- 
tion of certain features believed to be novel should not be 
construed as implying that other features arc not novcL 

The system provides for a database of assets which must 
be arranged temporally in order to be used, but which does 
not dictate the temporal organization. Rather, the audio and 
video assets in the database are characterized by such 
parameters as content, aesthetic features, and suitability to 
viewers with certain demographic or psychographic charac- 
teristics. The user can select assets through a suitable query 
without attention to temporal organization. Tlie organization 
of assets contrasts with the organization of assets in such 
fields as video games. Tn a video game, the permitted 
temporal orders of the assets are predetermined by a strict 
branching system. Such a branching system defines pre- 
cisely which asset may be permitted to follow which under 
all possible circumstances. The database of the present 
system may be termed amorphous, in contrast to the rigid 
organization of a database having a branching system. The 
general principle can be extended to other types of assets 
that require temporal organization in order to be used. For 
example, a database can be constructed for use with still 
photographs to be arranged into a program with an audio 
track. Interactive assets, such as two-dimensional or three- 
dimensional models and graphical user interfaces can also 
be catalogued in a database of this type. The present also 
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contrasts to traditional databases in that it mines non-linear 
assets for presentation in a linear manner, the linear manner 
not being contained within the assets. The use of a modera- 
tion layer that limits the assets responsive to the query that 

5 are actually presented to the user contrasts with traditional 
database organization, in which all items responsive to a 
query are presented to the user. 

The system also provides for the automatic creation of 
new programming, uniquely configured to the requirements 
of the viewer, but which may be viewed passively. In 
contrast, m such fields of video games, while the creation of 
unique programming occurs, the viewer must interact with 
the system in order to create the programming. 

The system is also capable of arranging assets in an order, 
and creating transitions between assets, to create a concat- 
cnatcd stream of audiovisual programming that has transi- 
tions that appear to the viewer to have been edited by a 
human editor. Even without temporal order, the assets con- 
tain sufiScient transition information, and the system con- 
tains default transition choices, to permit assembly of pro- 

20 gramming with fluid transitions without the intervention of 
a human editor. Numerous methods are available for order- 
ing the assets in such a program. As discussed, assets may 
be ordered by using a template that imposes sequential 
requirements on the assets, an expert system, a truth table, 

25 or a human editor may decide the order of the assets. ITie 
attributes of the assels impose certain Hmitalions on the 
ordering ot the assets. For example, it may not be possible 
for aesthetic reasons to place a certain ch*p immediately 
before or after a certain other clip. 

30 The individual assets may have demographic character- 
istics. For example, an audiovisual clip may feature actors of 
a particular ethnic group, geographic origin, age, or other 
demographic characteristic. As a result, the assets may be 
selected based on demographic search criteria. 

35 The system may permit, in an on-line environment, 
switching between a predetermined program and interactive 
video. For example, the system may provide delivery of a 
predetermined audiovisual program so long as the viewer 
does not seek to interact with the system, e.g., by posititjning 

40 a mouse pointer on the screen and clicking. At this point, the 
system may select assets that are appropriate to the particu- 
lar intbrmation that the viewer appears to be seeking based 
on the information on the screen at the time. The system will 
then generate suitable transitions and add those assets to the 

45 programming. For example, the program may be a tour of a 
three-dimensional model of the heart. When the user moves 
a pointer or mouse to a particular portion of the screen, and 
clicks on the screen, the system may select, for example, 
assets mcorporating more detailed information on certain 

50 portions of the heart corresponding to material on the 
portion of the screen where the mouse was located when 
clicked. The system applies suitable transitions to the assets 
and incorporates these assets into the programming interac- 
tively in response to mouse clicks or other suitable input 

55 firom the viewer. This interactive generation of programming 
for passive viewing contrasts with existing systems that 
include a predetermined loop if the viewer does not provide 
inputs. 

The system may also be employed in an interactive 
60 system that leads the viewer to select a desired sequence. 
TTie system may, for example, provide more interesting 
visual data in areas of the screen that lead to the selection of 
material that the viewer should see, or provide smooth 
transitions from audiovisual assets incorporated in the pro- 
65 gram as a result of viewer input to audiovisual assels 
incorporated in the program to achieve a predetermined 
purpose, e.g., to convey certain information to the viewer. 



